Soft Computing for Sensor and Algorithm Fusion

Sensor and algorithm fusion is playing an increasing role in many application domains. As detection and recognition problems become more complex and costly (for example, landmine detection and automatic target recognition), it is apparent that no single source of information can provide the ultimate solution. However, complementary information can be derived from multiple sources. Given a set of outputs from constituent sources, there are many frameworks within which to combine the pieces into a more definitive answer. This talk will focus on the fusion of multiple partial confidence values within the framework of fuzzy set theory. Take, for example, current research and development into the next generation of landmine detection systems. Almost all approaches include at least Ground Penetrating Radar (GPR), and Electromagnetic Inductance (EMI) metal detection sensors. Additionally, infrared (IR) imaging sensors and some form of chemical detection are increasing common. Within each of the individual sensor systems, features characterizing mines and background need to be developed, and classification algorithms applied to the feature sets. Since there is considerable uncertainty in the problem domain, it is unlikely that any single combination of feature set/classification algorithm will be sufficient to achieve high detection probability with very low false alarm rate under real conditions. (At least we haven’t found it yet). Our hope is that by using multiple algorithms and multiple sensors, even though strictly speaking they may be redundant, we can increase detection and lower false alarms. When different algorithms/sensors make different mistakes, fusion can make a real impact. So, the question then becomes: what methodology do we use to combine partial decision information? There are many choices, but I will focus on the use of fuzzy set theoretic mechanisms to fuse confidence from multiple sources. Two general approaches will be considered, fuzzy integrals and fuzzy logic rule-based systems. Fuzzy integrals have a long history and have been studied in the context of pattern recognition and information fusion for several years being first introduced for this purpose by Tahani and Keller in 1990. Fuzzy integrals combine the objective evidence supplied by each information source with the expected worth of each subset of information sources (via a fuzzy measure) to assign confidence to hypotheses or to rank alternatives in decision making. This is a nonlinear combination of information and the worth of the information for the decision in question, dealing with the uncertainty in both forms of data. Different fuzzy measures yield different integration operations, including averaging, order statistics (such as median, max, and min), linear combinations of order statistics, and many others. Measures can be found by heuristic assignment or via training algorithms. I will describe several projects were multiple classifiers outputs were combined with Sugeno and Choquet fuzzy integrals to produce significant increases in performance on complex real data sets from landmine detection and automatic target recognition. We have also used this approach to fuse classifiers for E. coli O157:H7 bacteria detection. Next, a fusion system based on a linguistic extension of the Choquet fuzzy integral will be shown. The uncertainty in the data is now expressed as a linguistic vector, i.e., a vector of fuzzy sets. The linguistic Choquet integral is used to fuse both position and confidence uncertainty in the landmine detection scenario. We test this system on the outputs from three algorithms on data collected from an outdoor test site by the GEO-CENTERS Energy Focusing Ground Penetrating Radar (EFGPR). The results show good improvement in the probability of detection and a reduction in the false alarm rate over the best single algorithm and two simple numeric fusion schemes, i.e., the confidence average and the numeric Choquet fuzzy integral. Fuzzy logic rule-based systems provide another mechanism to fuse together the results of different features, classification algorithms and sensors. Such a system employs rules much like those that a human expert might derive. Again, uncertainty in the component parts is modeled by linguistic variables taking on fuzzy sets as values. We first describe the application of a fuzzy rule-based classifier in E. coli recognition. While the fuzzy rule-based system achieved good recognition, the “human inspired” features used in the rules were incorporated into a multiple neural network fusion approach that gave excellent separation of the target bacteria. For landmine detection, we found that a fuzzy rule-based system (determined heuristically) for the fusion of classifier outputs produced the best score in blind testing. A more fundamental question, though, is the following. If we know the general characteristics of a set of sensors, can we predict the value added by fusing their outputs together? Correspondingly, can we specify the needed characteristics of a new sensor/algorithm to add to an existing suite to gain a desired improvement in performance? These questions are difficult and, of course, coupled to the fusion framework. As a preliminary step in this direction, we looked at a quantitative analysis of sensor system fusion of landmine detection locations using our linguistic paradigm. We developed new tools to examine the performance of detection position errors, modeled by vectors of fuzzy sets, in a simulation environment. The approach is shown with general data obtained from an actual test situation.